Conventional and known optical monitoring systems for buildings, stores, etc. have video cameras (e.g., security cameras) that form and transmit images to a security center, which is sometimes located at a remote location. In this type of monitoring system, human observers in the security center constantly view and monitor the transmitted monitor images to detect any monitored subject. In a security application, for example, the monitored subject could be a human intruder. A problem with such systems is that they require a large staff of human observers.
Also known are automated optical monitoring systems that use infrared sensors, etc. to detect infrared energy generated by or emitted from the monitored subjects (e.g., intruders). These automated monitoring devices minimize the personnel burden, but they can also detect small animals, such as rats, thereby having the problem of easily generating false alarms.
Furthermore, optical monitoring systems have been considered for automatically identifying intruders with image recognition to detect a monitored subject in a monitored region. However, the size of an intruder on the image plane varies greatly according to the imaging distance. Hence, these systems suffer from the problem that intruders cannot be uniformly image-recognized.
Therefore, an object of the present invention is to provide an optical monitoring system capable of automatically identifying whether a moving body is the monitored subject based upon information in the imaged image.
In one embodiment, the invention includes a moving body detection sub-system that images a monitored region and detects a moving body from changes over time in the image of the monitored region. A speed detection sub-system detects the speed of a moving body in the image plane (i.e., speed of the moving body image in the image plane). A scale detection sub-system detects the size of a moving body in the image plane (i.e., size of the moving body in the image plane). A moving body estimation sub-system decides whether a moving body is the monitored subject (e.g., a human being) based on the image plane speed detected by the speed detection sub-system and the image plane size detected by the scale detection sub-system.
The “image plane speed” and “image plane size” referred to above are relative parameters that change together depending on the imaging distance to the moving body. That is, when the imaging distance is large, the image plane size of the moving body is small. When this happens the moving body's image plane speed is also a small percentage, like the reduction percentage of the image plane size.
Therefore it is possible to easily cancel the effect of imaging distance included in both parameters by performing processing to find the ratio between image plane speed and image plane size, for example. By reducing the effect of, or accommodating, imaging distance based upon image information in this way, the moving body estimation sub-system of the present invention can identify a moving body without being affected by the variations in image plane size due to imaging distance.
In another embodiment, the moving body estimation sub-system has an actual scale estimation sub-system that estimates the actual size of a moving body based on the image plane speed detected by the speed detection sub-system and the image plane size detected by the scale detection sub-system. The moving body estimation sub-system decides whether a moving body is the monitored subject based on the estimated moving body's actual size.
In this embodiment, the actual scale estimation sub-system uses an image plane ruler (i.e., length standard) for movement speed of the moving body and converts the image plane size of the moving body into an actual size that does not depend on the imaging distance. The moving body estimation sub-system decides whether a moving body is the monitored subject based on this actual size, and is essentially unaffected by imaging distance.
In another embodiment, the moving body estimation sub-system has a correlation relationship storage sub-system that stores the correlation relationships between the image plane speed and image plane size of assumed moving bodies. In addition, the correlation relationship storage sub-system has a class estimation sub-system that checks the image plane speed detected by the speed detection sub-system and the image plane size detected by the scale detection sub-system against the correlation relationships stored in the correlation relationship storage sub-system. The correlation relationship storage sub-system then estimates the class of the moving body and decides whether the moving body is the monitored subject based on the estimated class of the moving body.
For example, if the “image plane speed” to “image plane size” correlation relationship of a nimble animal such as a cockroach or bee were applied as-is to the height of a human being, that movement speed would be a speed that far surpassed the world record in the sprint (about 2 seconds for a 100-meter run). This illustrates that the “image plane speed” to “image plane size” correlation relationship varies according to the class of animal.
In this embodiment, the correlation relationship storage sub-system stores “image plane speed” to “image plane size” correlation relationships for assumed moving body classes. The class estimation sub-system estimates whether a moving body belongs to an assumed animal class by checking the image plane speed and image plane size found from the moving body image against this correlation relationship. With this sort of estimation operation it possible to identify a moving body with almost no effect from differences in image plane size due to imaging distance.
In another embodiment, the moving body estimation sub-system has a moving body evaluation sub-system that calculates an evaluation value indicating the certainty that the moving body is the monitored subject. The evaluation value is based on the image plane speed detected by the speed detection sub-system and the image plane size detected by the scale detection sub-system. The moving body estimation sub-system decides whether the moving body is the monitored subject based on the evaluation value of the moving body evaluation sub-system.
The “image plane speed” to “image plane size” correlation relationship varies according to the class of animal. Therefore it is possible to calculate an evaluation value indicating the certainty of being the monitored subject by evaluating the extent to which these two parameters match the correlation relationship of the monitored subject. Therefore, in this embodiment, a moving body is identified as the monitored subject or not based on the calculated evaluation value. This embodiment makes it possible to identify a moving body with almost no effect from differences in image plane size due to imaging distance.
In another embodiment, a moving body detection sub-system images a monitored region and detects a moving body from changes over time in the monitored region. A position detection sub-system detects the position of the moving body in the image plane. A scale detection sub-system detects the size of the moving body in the image plane. A moving body estimation sub-system decides whether the moving body is the monitored subject based on the image plane position (i.e., position of the moving body image in the image plane) detected by the position detection sub-system and the image plane size (i.e., size of the moving body image in the image plane) detected by the scale detection sub-system.
In general, the image plane position found in this manner exhibits specific tendencies according to the class of animal. For example, if it is a human being there is a high possibility it will be positioned on a path in the screen. If it is a cockroach, it is not limited to a path; it may also be positioned on a wall surface. Therefore if the image plane position is on a wall surface, the moving body can be estimated to be a cockroach or the like, not a human being. Also, even if a moving body is positioned on a path, a human being and a cockroach clearly differ with regard to image plane size. Therefore it is possible to focus in to some extent on the class of animal and make an appropriate moving body identification based on two pieces of information—image plane position and image plane size.
As an alternative in the immediately preceding embodiment, the moving body estimation sub-system may have an actual scale estimation sub-system that estimates the actual size of the moving body based on the image plane position detected by the position detection sub-system and the image plane size detected by the scale detection sub-system. In this alternative, the moving body estimation sub-system decides whether the moving body is the monitored subject based on the estimated actual size of the moving body.
In another alternative in the immediately preceding embodiment, the moving body estimation sub-system has a correlation relationship storage sub-system that stores the correlation relationships between the image plane position and image plane size of assumed moving bodies. A class estimation sub-system checks the image plane position detected by the position detection sub-system and the image plane size detected by the scale detection sub-system against the correlation relationships stored in the correlation relationship storage sub-system. The class estimation sub-system estimates the class of the moving body and decides whether the moving body is the monitored subject based on the estimated class of the moving body.
When a monitored region is determined and an imaged region is observed for a long period of time, unique tendencies appear with regard to the image plane positions that are passed through and the image plane size, and these tendencies can vary according to the class of moving body. Therefore this sort of tendency is found as a correlation relationship through statistical processing, etc., and stored in advance in the correlation relationship storage sub-system.
The class estimation sub-system checks the image plane position and image plane size of the current moving body against the correlation relationships stored in the correlation relationship storage sub-system. The class estimation sub-system estimates the class of the moving body based on the extent of the comparison. Through this sort of operation the present invention makes it possible to identify a moving body with almost no effect from differences in image plane size due to imaging distance.
In another alternative in the immediately preceding embodiment, the moving body estimation sub-system has a moving body evaluation sub-system that calculates an evaluation value indicating the certainty that the moving body is the monitored subject based on the image plane position detected by the position detection sub-system and the image plane size detected by the scale detection sub-system. The moving body estimation sub-system decides whether the moving body is the monitored subject based on the evaluation value of the moving body evaluation sub-system.
The “image plane position” to “image plane size” correlation relationship exhibits specific tendencies according to the class of moving body. Therefore it is possible to calculate an evaluation value indicating the certainty of being the monitored subject by evaluating the extent to which the detected image plane position and image plane size match the correlation relationship of the monitored subject. In this alternative, a moving body is identified as the monitored subject based on the evaluation value calculated in this way. This makes it possible to identify a moving body with almost no effect from differences in image plane size due to imaging distance.
As an alternative in any of the preceding embodiments, the moving body estimation sub-system decides whether a moving body is the monitored subject for a limited specified area of the monitored region.
In this alternative, a moving body is identified for limited specified sites in the monitored region. By limiting specified sites in this manner it is possible to appropriately increase the decision accuracy. In addition, the range subject to moving body identification is limited, so it is possible to reduce the amount of calculation processing needed for identifying a moving body.
As a result, a monitoring system according to the present invention can automatically identify intruders or other moving things with high reliability. Also, a monitoring system according to can perform automatic identification using only image plane information, so separate distance-finding devices are not necessary, and the monitoring system structure can be simple and inexpensive.
Additional objects and advantages of the present invention will be apparent from the detailed description of the preferred embodiment thereof, which proceeds with reference to the accompanying drawings.